So the problem is we don't know what the state of the world is in.

If we have a simple environment that's deterministic, partially observable, static, and all the

other good stuff, then if we know the state now, we can compute the state in the future.

And we're always certain that it will be this way, otherwise it wouldn't be static.

If I drop this on my foot in a static world, it will actually find my foot.

So this problem is within uncertainty, with uncertain actions, with stochastic environments

and so on.

It doesn't hold anymore.

So we need to extend our world model.

So instead of keeping one world state, we need to keep a set of possible world states.

The Mensa might be open or it might be closed or it might even have burnt down.

We have no way of knowing.

So if we really want to be careful in the uncertain world that we have here, we have

to plan for all three.

We have three world states as pertaining to the Mensa.

But of course, and we believe that it could be in these three states, or we have reasoned,

well, if the Mensa burnt down, then we would probably have heard the fire engines coming.

And so we didn't hear that, so it's probably still standing there.

So we have two possible worlds.

And what we have there is a belief model, which essentially keeps a set of possible

world worlds, and a transition model that tells us about how these possible worlds evolve.

And that tells us what the next state set of possible worlds is.

And the fire engine is we know something about evolving of worlds with burning Mensas and

the number of fire engines that are coming and so on.

So since the transition of if the Mensa burns, then there will be loud fire engines, that

actually gets us to believe that actually there's only two states for the Mensa, open

or closed.

Because that's something we, if it just closes now, then we wouldn't hear that.

And we wouldn't notice it.

So we have to entertain those two things.

So in essence, if the environment is bad enough, dynamic or semi-dynamic or episodic or all

those kind of bad things, then we have to plan accordingly for our world model.

And typically, we need to have something like a belief state, what we believe the possible

set of possible worlds could be.

In a fully deterministic environment, we only need a singleton world model, belief state.

Because we know what the world is going to be, because we can compute it from the last

state and our sensors.

So let's see whether we can kind of match that to last semester's stuff.

In these search-based agents, we had a fully observable deterministic environment, meaning

if my action is to move the rook from A1 to D4, then that will actually succeed.

You don't see chess players trying to move your piece and then failing.

It's not what chess is like.

Soccer is like that.

I try to run really fast and get to the goal first, and then I don't.

So we have a fully observable deterministic environment.

So the world state or the world model is just the current state, which we can actually observe.

Easy peasy.

A reflex agent can do that.

You don't even have to keep the T. CSP is a fully observable, still deterministic environment,

but we're not keeping the actual state, but a set of constraints that describe the possible